Pandas Profiling Report

Dataset statistics

Number of variables	11
Number of observations	10000
Missing cells	99
Missing cells (%)	0.1%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	859.5 KiB
Average record size in memory	88.0 B

Variable types

Numeric	10
Categorical	1

Alerts

`longitude` is highly correlated with `latitude`	High correlation
`latitude` is highly correlated with `longitude`	High correlation
`total_rooms` is highly correlated with `total_bedrooms` and 2 other fields	High correlation
`total_bedrooms` is highly correlated with `total_rooms` and 2 other fields	High correlation
`population` is highly correlated with `total_rooms` and 2 other fields	High correlation
`households` is highly correlated with `total_rooms` and 2 other fields	High correlation
`median_income` is highly correlated with `median_house_value`	High correlation
`median_house_value` is highly correlated with `median_income`	High correlation
`longitude` is highly correlated with `latitude`	High correlation
`latitude` is highly correlated with `longitude`	High correlation
`total_rooms` is highly correlated with `total_bedrooms` and 2 other fields	High correlation
`total_bedrooms` is highly correlated with `total_rooms` and 2 other fields	High correlation
`population` is highly correlated with `total_rooms` and 2 other fields	High correlation
`households` is highly correlated with `total_rooms` and 2 other fields	High correlation
`median_income` is highly correlated with `median_house_value`	High correlation
`median_house_value` is highly correlated with `median_income`	High correlation
`longitude` is highly correlated with `latitude`	High correlation
`latitude` is highly correlated with `longitude`	High correlation
`total_rooms` is highly correlated with `total_bedrooms` and 2 other fields	High correlation
`total_bedrooms` is highly correlated with `total_rooms` and 2 other fields	High correlation
`population` is highly correlated with `total_rooms` and 2 other fields	High correlation
`households` is highly correlated with `total_rooms` and 2 other fields	High correlation
`df_index` is highly correlated with `longitude` and 3 other fields	High correlation
`longitude` is highly correlated with `df_index` and 3 other fields	High correlation
`latitude` is highly correlated with `df_index` and 3 other fields	High correlation
`total_rooms` is highly correlated with `total_bedrooms` and 2 other fields	High correlation
`total_bedrooms` is highly correlated with `total_rooms` and 2 other fields	High correlation
`population` is highly correlated with `total_rooms` and 2 other fields	High correlation
`households` is highly correlated with `total_rooms` and 2 other fields	High correlation
`median_income` is highly correlated with `median_house_value`	High correlation
`median_house_value` is highly correlated with `df_index` and 4 other fields	High correlation
`ocean_proximity` is highly correlated with `df_index` and 3 other fields	High correlation
`df_index` has unique values	Unique

Reproduction

Analysis started	2022-03-15 18:09:30.378533
Analysis finished	2022-03-15 18:09:39.540449
Duration	9.16 seconds
Software version	pandas-profiling v3.1.0
Download configuration	config.json

df_index
Real number (ℝ_≥0)

HIGH CORRELATION
UNIQUE

Distinct	10000
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	10314.8348

Minimum	0
Maximum	20639
Zeros	1
Zeros (%)	< 0.1%
Negative	0
Negative (%)	0.0%
Memory size	78.2 KiB

Quantile statistics

Minimum	0
5-th percentile	980.95
Q1	5114.25
median	10277
Q3	15532.25
95-th percentile	19602.05
Maximum	20639
Range	20639
Interquartile range (IQR)	10418

Descriptive statistics

Standard deviation	5983.435059
Coefficient of variation (CV)	0.5800805514
Kurtosis	-1.205372537
Mean	10314.8348
Median Absolute Deviation (MAD)	5210.5
Skewness	-0.002737784802
Sum	103148348
Variance	35801495.1
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
6141	1	< 0.1%
9614	1	< 0.1%
5528	1	< 0.1%
11671	1	< 0.1%
9622	1	< 0.1%
3475	1	< 0.1%
17810	1	< 0.1%
5520	1	< 0.1%
11663	1	< 0.1%
15757	1	< 0.1%
Other values (9990)	9990	99.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	1	< 0.1%
3	1	< 0.1%
4	1	< 0.1%
7	1	< 0.1%
8	1	< 0.1%
9	1	< 0.1%
11	1	< 0.1%
12	1	< 0.1%
13	1	< 0.1%
15	1	< 0.1%

Value	Count	Frequency (%)
20639	1	< 0.1%
20637	1	< 0.1%
20632	1	< 0.1%
20631	1	< 0.1%
20630	1	< 0.1%
20629	1	< 0.1%
20625	1	< 0.1%
20623	1	< 0.1%
20620	1	< 0.1%
20619	1	< 0.1%

longitude
Real number (ℝ)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	766
Distinct (%)	7.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	-119.599309

Minimum	-124.35
Maximum	-114.31
Zeros	0
Zeros (%)	0.0%
Negative	10000
Negative (%)	100.0%
Memory size	78.2 KiB

Quantile statistics

Minimum	-124.35
5-th percentile	-122.46
Q1	-121.82
median	-118.55
Q3	-118.02
95-th percentile	-117.08
Maximum	-114.31
Range	10.04
Interquartile range (IQR)	3.8

Descriptive statistics

Standard deviation	2.00345197
Coefficient of variation (CV)	-0.01675136743
Kurtosis	-1.360759623
Mean	-119.599309
Median Absolute Deviation (MAD)	1.32
Skewness	-0.2628071996
Sum	-1195993.09
Variance	4.013819794
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-118.29	80	0.8%
-118.3	78	0.8%
-118.31	72	0.7%
-118.35	70	0.7%
-118.28	69	0.7%
-118.36	68	0.7%
-118.34	67	0.7%
-118.32	67	0.7%
-118.14	64	0.6%
-118.13	63	0.6%
Other values (756)	9302	93.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-124.35	1	< 0.1%
-124.3	1	< 0.1%
-124.27	1	< 0.1%
-124.25	1	< 0.1%
-124.23	1	< 0.1%
-124.22	1	< 0.1%
-124.21	3	< 0.1%
-124.19	1	< 0.1%
-124.18	4	< 0.1%
-124.17	7	0.1%

Value	Count	Frequency (%)
-114.31	1	< 0.1%
-114.47	1	< 0.1%
-114.49	1	< 0.1%
-114.55	1	< 0.1%
-114.56	1	< 0.1%
-114.57	1	< 0.1%
-114.59	1	< 0.1%
-114.6	2	< 0.1%
-114.62	1	< 0.1%
-114.63	1	< 0.1%

latitude
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	794
Distinct (%)	7.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	35.658314

Minimum	32.55
Maximum	41.95
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	78.2 KiB

Quantile statistics

Minimum	32.55
5-th percentile	32.83
Q1	33.94
median	34.28
Q3	37.73
95-th percentile	38.98
Maximum	41.95
Range	9.4
Interquartile range (IQR)	3.79

Descriptive statistics

Standard deviation	2.137125726
Coefficient of variation (CV)	0.0599334485
Kurtosis	-1.138330753
Mean	35.658314
Median Absolute Deviation (MAD)	1.29
Skewness	0.4441786788
Sum	356583.14
Variance	4.567306368
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
34.06	125	1.2%
34.02	110	1.1%
34.08	109	1.1%
34.05	108	1.1%
34.04	107	1.1%
34.07	107	1.1%
33.97	99	1.0%
34.09	99	1.0%
33.94	98	1.0%
34.03	94	0.9%
Other values (784)	8944	89.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
32.55	3	< 0.1%
32.56	5	0.1%
32.57	6	0.1%
32.58	10	0.1%
32.59	4	< 0.1%
32.6	2	< 0.1%
32.61	3	< 0.1%
32.62	8	0.1%
32.63	10	0.1%
32.64	11	0.1%

Value	Count	Frequency (%)
41.95	1	< 0.1%
41.92	1	< 0.1%
41.88	1	< 0.1%
41.86	1	< 0.1%
41.84	1	< 0.1%
41.81	1	< 0.1%
41.8	1	< 0.1%
41.79	1	< 0.1%
41.78	2	< 0.1%
41.77	1	< 0.1%

housing_median_age
Real number (ℝ_≥0)

Distinct	52
Distinct (%)	0.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	28.5256

Minimum	1
Maximum	52
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	78.2 KiB

Quantile statistics

Minimum	1
5-th percentile	8
Q1	18
median	28
Q3	37
95-th percentile	52
Maximum	52
Range	51
Interquartile range (IQR)	19

Descriptive statistics

Standard deviation	12.6589482
Coefficient of variation (CV)	0.4437750021
Kurtosis	-0.8019724061
Mean	28.5256
Median Absolute Deviation (MAD)	10
Skewness	0.07901039379
Sum	285256
Variance	160.2489695
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
52	648	6.5%
36	385	3.9%
35	381	3.8%
16	370	3.7%
17	354	3.5%
34	338	3.4%
33	313	3.1%
26	292	2.9%
25	278	2.8%
18	274	2.7%
Other values (42)	6367	63.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	4	< 0.1%
2	27	0.3%
3	28	0.3%
4	98	1.0%
5	115	1.1%
6	77	0.8%
7	85	0.9%
8	112	1.1%
9	110	1.1%
10	136	1.4%

Value	Count	Frequency (%)
52	648	6.5%
51	27	0.3%
50	52	0.5%
49	57	0.6%
48	97	1.0%
47	99	1.0%
46	118	1.2%
45	131	1.3%
44	169	1.7%
43	165	1.7%

total_rooms
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	4504
Distinct (%)	45.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	2639.8222

Minimum	6
Maximum	37937
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	78.2 KiB

Quantile statistics

Minimum	6
5-th percentile	619.95
Q1	1440
median	2123.5
Q3	3155
95-th percentile	6200.05
Maximum	37937
Range	37931
Interquartile range (IQR)	1715

Descriptive statistics

Standard deviation	2207.480098
Coefficient of variation (CV)	0.836223022
Kurtosis	34.9027477
Mean	2639.8222
Median Absolute Deviation (MAD)	799.5
Skewness	4.293792926
Sum	26398222
Variance	4872968.381
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
1613	11	0.1%
2053	10	0.1%
1562	9	0.1%
1387	9	0.1%
1729	9	0.1%
1999	9	0.1%
1834	9	0.1%
1548	9	0.1%
1722	9	0.1%
1471	9	0.1%
Other values (4494)	9907	99.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
6	1	< 0.1%
11	1	< 0.1%
12	1	< 0.1%
15	1	< 0.1%
16	1	< 0.1%
18	3	< 0.1%
19	2	< 0.1%
20	1	< 0.1%
21	1	< 0.1%
25	1	< 0.1%

Value	Count	Frequency (%)
37937	1	< 0.1%
32627	1	< 0.1%
32054	1	< 0.1%
30405	1	< 0.1%
30401	1	< 0.1%
27700	1	< 0.1%
26322	1	< 0.1%
25957	1	< 0.1%
23915	1	< 0.1%
23866	1	< 0.1%

total_bedrooms
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	1582
Distinct (%)	16.0%
Missing	99
Missing (%)	1.0%
Infinite	0
Infinite (%)	0.0%
Mean	537.3978386

Minimum	2
Maximum	6445
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	78.2 KiB

Quantile statistics

Minimum	2
5-th percentile	136
Q1	294
median	434
Q3	647
95-th percentile	1259
Maximum	6445
Range	6443
Interquartile range (IQR)	353

Descriptive statistics

Standard deviation	424.5896795
Coefficient of variation (CV)	0.790084457
Kurtosis	23.70419168
Mean	537.3978386
Median Absolute Deviation (MAD)	162
Skewness	3.583189769
Sum	5320776
Variance	180276.396
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
280	30	0.3%
287	30	0.3%
300	30	0.3%
416	27	0.3%
361	27	0.3%
460	27	0.3%
254	27	0.3%
272	26	0.3%
289	26	0.3%
291	25	0.2%
Other values (1572)	9626	96.3%
(Missing)	99	1.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
2	1	< 0.1%
3	4	< 0.1%
4	5	0.1%
5	4	< 0.1%
6	1	< 0.1%
7	2	< 0.1%
8	7	0.1%
9	3	< 0.1%
10	4	< 0.1%
11	4	< 0.1%

Value	Count	Frequency (%)
6445	1	< 0.1%
5471	1	< 0.1%
5290	1	< 0.1%
4957	1	< 0.1%
4952	1	< 0.1%
4798	1	< 0.1%
4585	1	< 0.1%
4492	1	< 0.1%
4407	1	< 0.1%
4335	1	< 0.1%

population
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	3077
Distinct (%)	30.8%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	1422.4701

Minimum	5
Maximum	28566
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	78.2 KiB

Quantile statistics

Minimum	5
5-th percentile	349
Q1	781
median	1164
Q3	1728
95-th percentile	3269.45
Maximum	28566
Range	28561
Interquartile range (IQR)	947

Descriptive statistics

Standard deviation	1128.547999
Coefficient of variation (CV)	0.7933720357
Kurtosis	53.22359773
Mean	1422.4701
Median Absolute Deviation (MAD)	444
Skewness	4.515762475
Sum	14224701
Variance	1273620.586
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
999	14	0.1%
891	14	0.1%
825	13	0.1%
761	13	0.1%
784	12	0.1%
990	12	0.1%
899	12	0.1%
746	12	0.1%
984	12	0.1%
1047	12	0.1%
Other values (3067)	9874	98.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
5	1	< 0.1%
8	4	< 0.1%
9	2	< 0.1%
13	2	< 0.1%
15	1	< 0.1%
17	2	< 0.1%
18	1	< 0.1%
19	1	< 0.1%
20	1	< 0.1%
21	1	< 0.1%

Value	Count	Frequency (%)
28566	1	< 0.1%
16122	1	< 0.1%
15507	1	< 0.1%
15037	1	< 0.1%
13251	1	< 0.1%
12873	1	< 0.1%
12153	1	< 0.1%
11973	1	< 0.1%
11272	1	< 0.1%
10877	1	< 0.1%

households
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	1487
Distinct (%)	14.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	499.323

Minimum	2
Maximum	6082
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	78.2 KiB

Quantile statistics

Minimum	2
5-th percentile	124
Q1	277
median	409
Q3	605
95-th percentile	1143.1
Maximum	6082
Range	6080
Interquartile range (IQR)	328

Descriptive statistics

Standard deviation	386.1194174
Coefficient of variation (CV)	0.7732858639
Kurtosis	24.88322346
Mean	499.323
Median Absolute Deviation (MAD)	152
Skewness	3.59569574
Sum	4993230
Variance	149088.2045
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
309	30	0.3%
362	29	0.3%
428	29	0.3%
239	28	0.3%
306	28	0.3%
375	28	0.3%
267	28	0.3%
265	27	0.3%
274	27	0.3%
369	27	0.3%
Other values (1477)	9719	97.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
2	1	< 0.1%
3	3	< 0.1%
4	3	< 0.1%
5	3	< 0.1%
6	3	< 0.1%
7	4	< 0.1%
8	5	0.1%
9	6	0.1%
10	2	< 0.1%
11	2	< 0.1%

Value	Count	Frequency (%)
6082	1	< 0.1%
5189	1	< 0.1%
5050	1	< 0.1%
4616	1	< 0.1%
4490	1	< 0.1%
4372	1	< 0.1%
4339	1	< 0.1%
4176	1	< 0.1%
4072	1	< 0.1%
4012	1	< 0.1%

median_income
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	7329
Distinct (%)	73.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	3.88756159

Minimum	0.4999
Maximum	15.0001
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	78.2 KiB

Quantile statistics

Minimum	0.4999
5-th percentile	1.6007
Q1	2.580775
median	3.5417
Q3	4.7708
95-th percentile	7.352105
Maximum	15.0001
Range	14.5002
Interquartile range (IQR)	2.190025

Descriptive statistics

Standard deviation	1.914270559
Coefficient of variation (CV)	0.4924090629
Kurtosis	5.227155227
Mean	3.88756159
Median Absolute Deviation (MAD)	1.0676
Skewness	1.68056668
Sum	38875.6159
Variance	3.664431774
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
15.0001	31	0.3%
4.125	25	0.2%
3.875	24	0.2%
3.125	23	0.2%
2.875	23	0.2%
4	20	0.2%
2.125	19	0.2%
4.625	19	0.2%
3.25	17	0.2%
3.375	16	0.2%
Other values (7319)	9783	97.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0.4999	7	0.1%
0.536	4	< 0.1%
0.5495	1	< 0.1%
0.6433	1	< 0.1%
0.6825	1	< 0.1%
0.696	1	< 0.1%
0.7007	1	< 0.1%
0.7025	1	< 0.1%
0.7068	1	< 0.1%
0.7069	1	< 0.1%

Value	Count	Frequency (%)
15.0001	31	0.3%
15	1	< 0.1%
14.5833	1	< 0.1%
13.947	1	< 0.1%
13.8093	1	< 0.1%
13.5728	1	< 0.1%
13.367	1	< 0.1%
13.2986	1	< 0.1%
13.1867	1	< 0.1%
13.1738	1	< 0.1%

median_house_value
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	3228
Distinct (%)	32.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	207235.5953

Minimum	14999
Maximum	500001
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	78.2 KiB

Quantile statistics

Minimum	14999
5-th percentile	67000
Q1	121075
median	179750
Q3	265100
95-th percentile	481570
Maximum	500001
Range	485002
Interquartile range (IQR)	144025

Descriptive statistics

Standard deviation	114892.0616
Coefficient of variation (CV)	0.5544031247
Kurtosis	0.3219888047
Mean	207235.5953
Median Absolute Deviation (MAD)	68150
Skewness	0.9734128898
Sum	2072355953
Variance	1.320018582 × 10¹⁰
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
500001	456	4.6%
137500	62	0.6%
162500	58	0.6%
112500	48	0.5%
87500	40	0.4%
187500	40	0.4%
225000	38	0.4%
275000	37	0.4%
350000	36	0.4%
175000	31	0.3%
Other values (3218)	9154	91.5%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
14999	3	< 0.1%
17500	1	< 0.1%
22500	3	< 0.1%
25000	1	< 0.1%
26900	1	< 0.1%
27500	1	< 0.1%
30000	2	< 0.1%
32500	1	< 0.1%
32900	1	< 0.1%
33200	1	< 0.1%

Value	Count	Frequency (%)
500001	456	4.6%
500000	9	0.1%
499100	1	< 0.1%
498800	1	< 0.1%
498700	1	< 0.1%
497400	1	< 0.1%
495800	1	< 0.1%
495600	1	< 0.1%
495500	2	< 0.1%
494400	1	< 0.1%

ocean_proximity
Categorical

HIGH CORRELATION

Distinct	5
Distinct (%)	0.1%
Missing	0
Missing (%)	0.0%
Memory size	78.2 KiB

<1H OCEAN	4418
INLAND	3165
NEAR OCEAN	1284
NEAR BAY	1131
ISLAND	2

Length

Max length	10
Median length	9
Mean length	8.0652
Min length	6

Characters and Unicode

Total characters	0
Distinct characters	0
Distinct categories	0 ?
Distinct scripts	0 ?
Distinct blocks	0 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	NEAR OCEAN
2nd row	INLAND
3rd row	<1H OCEAN
4th row	INLAND
5th row	<1H OCEAN

Common Values

Value	Count	Frequency (%)
<1H OCEAN	4418	44.2%
INLAND	3165	31.6%
NEAR OCEAN	1284	12.8%
NEAR BAY	1131	11.3%
ISLAND	2	< 0.1%

Length

Histogram of lengths of the category

Pie chart

Value	Count	Frequency (%)
ocean	5702	33.9%
1h	4418	26.2%
inland	3165	18.8%
near	2415	14.3%
bay	1131	6.7%
island	2	< 0.1%

Most occurring characters

Value	Count	Frequency (%)
No values found.

Most occurring categories

Value	Count	Frequency (%)
No values found.

Most frequent character per category

Most occurring scripts

Value	Count	Frequency (%)
No values found.

Most frequent character per script

Most occurring blocks

Value	Count	Frequency (%)
No values found.

Most frequent character per block

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

The dendrogram allows you to more fully correlate variable completion, revealing trends deeper than the pairwise ones visible in the correlation heatmap.

First rows

	df_index	longitude	latitude	housing_median_age	total_rooms	total_bedrooms	population	households	median_income	median_house_value	ocean_proximity
0	14486	-117.25	32.86	30.0	1670.0	219.0	606.0	202.0	12.4429	500001.0	NEAR OCEAN
1	2095	-119.76	36.75	35.0	1607.0	383.0	1407.0	382.0	2.1900	53400.0	INLAND
2	7522	-118.24	33.91	40.0	972.0	240.0	761.0	225.0	1.4688	88200.0	<1H OCEAN
3	20592	-121.58	39.14	52.0	662.0	160.0	520.0	149.0	0.8928	55000.0	INLAND
4	18166	-122.02	37.35	18.0	1221.0	255.0	507.0	271.0	5.3679	228400.0	<1H OCEAN
5	8344	-118.34	33.93	35.0	1213.0	284.0	742.0	253.0	4.0625	159900.0	<1H OCEAN
6	2433	-119.63	36.60	33.0	1589.0	294.0	1102.0	307.0	1.9676	62400.0	INLAND
7	3111	-117.68	35.62	30.0	2994.0	741.0	1481.0	581.0	2.1458	52400.0	INLAND
8	10408	-117.58	33.65	4.0	2000.0	422.0	833.0	386.0	5.7709	190300.0	<1H OCEAN
9	18033	-121.94	37.24	26.0	2561.0	388.0	1165.0	393.0	7.3522	363800.0	<1H OCEAN

Last rows

	df_index	longitude	latitude	housing_median_age	total_rooms	total_bedrooms	population	households	median_income	median_house_value	ocean_proximity
9990	15702	-122.44	37.79	52.0	3785.0	808.0	1371.0	799.0	6.4209	500001.0	NEAR BAY
9991	19486	-120.98	37.67	33.0	1433.0	298.0	824.0	302.0	2.7621	109100.0	INLAND
9992	7311	-118.20	33.99	31.0	1186.0	387.0	2087.0	409.0	1.9132	154600.0	<1H OCEAN
9993	1180	-121.54	39.47	14.0	1724.0	315.0	939.0	302.0	2.4952	53900.0	INLAND
9994	5202	-118.28	33.94	44.0	1631.0	338.0	1197.0	355.0	3.0788	100000.0	<1H OCEAN
9995	4806	-118.34	34.02	49.0	1609.0	371.0	896.0	389.0	2.5156	136600.0	<1H OCEAN
9996	7040	-118.09	33.94	36.0	2762.0	472.0	1576.0	493.0	4.0846	183400.0	<1H OCEAN
9997	9939	-122.28	38.22	42.0	106.0	18.0	40.0	25.0	7.5197	275000.0	NEAR BAY
9998	13314	-117.65	34.07	52.0	1041.0	252.0	558.0	231.0	1.9236	117200.0	INLAND
9999	134	-122.19	37.83	28.0	1326.0	184.0	463.0	190.0	8.2049	335200.0	NEAR BAY

Overview

Variables

Common Values

Length

Pie chart

Most occurring characters

Most occurring categories

Most frequent character per category

Most occurring scripts

Most frequent character per script

Most occurring blocks

Most frequent character per block

Interactions

Correlations

Spearman's ρ

Pearson's r

Kendall's τ

Phik (φk)

Missing values

Sample

First rows

Last rows